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Poor mathematics achievement in middle school students is evident in many countries. 
While some of the difficulties can be attributed to student related factors, there is 
considerable evidence that computational automaticity is essential for mathematics 
achievement. A QuickSmart (OS) mathematics intervention program was trialled with 
a group of students in Grades 7 and 8, matched with a control group of similar 
underachieving classmates. A statistically significant decrease in mean response 
latencies was found for OS participants after lessons in multiplication only. Significant 
differences were also evident between the pre and post scores of the two groups on a 
standardised test of mathematics. This study confirms and extends previous findings of 
the efficacy of mathematics intervention for underperforming middle school students. 


INTRODUCTION 


Poor achievement in mathematics has been investigated in numerous studies in many 
countries in relation to student factors such as cognitive difficulties, memory, attention, 
motivation, anxiety and self efficacy. Other studies have focussed on teaching methods 
and curriculum issues (Vaughn, Bos & Schumm, 2000), with dyspedagogia 
(Westwood, 2004) or poor teaching cited as having a significant impact on student 
failure in basic mathematics. While the development of mathematical reasoning 
depends on students learning appropriate facts, concepts, strategies and beliefs, lack of 
procedural knowledge of the basic operations for addition, subtraction, multiplication 
and division 1s the most obvious obstacle to academic success in mathematics (Mayer, 
2006, p. 65). The development of computational fluency or the speed with which 
students can retrieve or calculate answers to simple mathematics problems is a 
prerequisite to mathematics achievement at all levels (Arroyo, Royer & Woolf, 2011). 
Cognitive psychologists have established a clear relationship between the development 
of basic computational automaticity and complex mathematical problem solving skills 
(Tronsky & Royer, 2002). A multiplicity of studies have found that being able to 
produce answers to basic number facts rapidly and accurately reduces the load on the 
working memory and it is this saving that is a key factor in being able to develop more 
complex problem solving abilities (Tronsky & Royer, 2002, p. 118). 


A recent report on Australia’s performance in the 2012 Programme for International 
Student Assessment (PISA) has raised considerable concerns about the significant 
decline in the mathematical literacy of 15 year old students in Australia in general and 
South Australia in particular (Thomson, De Bortoli & Buckley, 2013). Between PISA 
2000 and PISA 2012 Australia’s mean mathematical literacy performance dropped 
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significantly from fifth to 19" place, with the decline most evident in the mean increase 
in the proportion of low performing students and decrease in top performers. South 
Australia has experienced one of the largest deteriorations, with the decrease of 46 
score points equivalent to more than a year of schooling and where 12 per cent more 
students did not reach base Level 2 in 2012 (Thomson, Hillman & De Bortoli, 2013). 
Significant declines in South Australia are also evident in Grade 8 students in the 
Trends in International Mathematics and Science Study (TIMSS) from 1995 to 2011 
(Thomson, Hillman & Wernert, 2012). Results from TIMSS 2011 have highlighted a 
substantial ‘tail’ of underperformance in mathematics, with 11 per cent of Australian 
students not even achieving the Low international benchmark (Thomson et al., 2012). 


Students’ poor performance in mathematics poses significant pedagogical issues for 
schools but particularly for middle school teachers who are caught in a “back to basics” 
dilemma (Yates, 2009a). Over time underachieving students fall increasingly behind 
their normally achieving peers and by the eighth year of school can be up to five years 
behind their average achieving peers (Pegg & Graham, 2007). Many students in the 
middle school years have to expend considerable effort to work on lower level 
component skills they have encountered many times before (Hattie & Yates, 2014). 
Practice is essential for students to gain automaticity of basic skills in mathematics 
content areas (National Mathematics Advisory Panel, 2008), but in middle school 
classrooms there is much less time and opportunity to develop skills that should have 
acquired in the early elementary grades (Carr, Taasoobshirazi, Stroud & Royer, 2011). 


THE PRESENT STUDY 


This research report is part of a larger longitudinal study of student mathematics 
achievement, self efficacy, anxiety and learned helplessness in a non-government, 
single sex, elementary and secondary school for boys in Adelaide, South Australia. 
The Progressive Achievement Tests in Mathematics Plus (PATM) (Australian Council 
for Educational Research, 2010) were administered online to students in Grades 3 to 10 
in March (Time 1) (T1) and November (Time 2) (T2), 2013. At T1 students completed 
the PATM test for their respective previous Grade level and at T2 their current Grade. 
At T1 a ‘tail’ of underperformance (Thomson et al., 2012) was most evident in Middle 
School students, with 33 (53%) of 63 Grade 7 boys and 33 (24%) of 140 Grade 8 boys 
scoring in the percentile rank range of 1-19 (-1 standard deviation) (SD). A further nine 
students in Grade 7 and 17 students in Grade 8 scored in the 20-30 percentile rank 
range. The poor achievement of some students could be accounted for, at least in part, 
by a verified learning disability or difficulty, but individual PATM profiles for the 
remaining students indicated poor performance in the numeracy strand. 


The school decided to trial the research-based QuickSmart (QS) mathematics 
intervention for the latter group as the program is designed to improve low achieving 
middle school students automaticity with addition, subtraction, multiplication and 
division over 30 weeks (Pegg, Graham, & Bellert, 2005; Graham & Pegg, 2010). 
Previous studies have shown QS participants gained on average two to three years 
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progress (effect size 0.49 - 0.80) measured from PATM pre to post test scores (Pegg & 
Graham, 2009) and improved on measures of response speed and accuracy compared 
with average achieving (Bellert, 2009) or high achieving same-age non-participants 
(Graham, Bellert, Thomas & Pegg, 2007). The present quasi-experimental study used 
pairs of underperforming students in the same mathematics classrooms to compare pre 
and post scores on the PATM. Further, the comparisons between QS participants and 
the paired classmates were undertaken after the completion of the first part of the QS 
program on multiplication rather than at its conclusion as in the previous studies. 
Response automaticity and accuracy were examined for both groups prior to QS and 
for QS students at the completion of the multiplication section of the intervention. 


AIMS 


1. To investigate the QS intervention program for middle school students with poor 
achievement in mathematics; and 

2. To compare the performance of students participating in the QS program with 
their paired classmates who received classroom instruction in mathematics only. 


METHOD 
Participants 


Eight Grade 7 students and 12 Grade 8 students with PATM scores in the 1-30 
percentile rank range at Tl were nominated by their mathematics teachers on the basis 
of their attendance, behaviour and poor performance on the PATM numeracy subscale. 
The 20 students were paired within their mathematics classes, with one student from 
each pair assigned to the QS group and the other to a control group. QS students were 
then grouped in pairs by Grade level for the delivery of the program. Students ranged 
in age from 12.3 years to 13.11 years with a median age of 13.3 years. 


The QuickSmart Mathematics Intervention Program (Graham et al., 2007) 


The multiplication section of QS was delivered to pairs of students in three 30 minute 
lessons per week over a mean of 16.5 hours. 


Procedure 


In Terms 3 and 4 a mean of 33 lessons, focussed on multiplication only, were delivered 
to the 10 pairs of designated QS students by a trained teacher aide, supervised by a 
registered teacher. These lessons were additional to their classroom instruction in 
mathematics. Each QS lesson consisted of 5 minute sections of a knowledge and 
understanding check, flashcards, speed sheet challenge of multiplication number facts, 
independent work sheet/strategy development, assessment and games. Response 
speed and accuracy was measured separately for each student in each lesson with the 
Cognitive Aptitude Assessment System (CASS) computer package (Royer, 1996) 
which is based on the Baddeley model of working memory (Tronsky & Royer, 2002). 
The CASS times student verbal responses via a microphone to randomised number 
sentences on a computer screen while the aide scores each response for accuracy. 
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Results are averaged and graphed automatically, providing each student with feedback 
to monitor his performance immediately and over time. QS and control students were 
tested with the CASS prior to the commencement of the intervention in Term 3, but 
thereafter students in the control group received five 40 minute mathematics lessons 
per week only. Numeracy achievement data from the National Assessment Program — 
Literacy and Numeracy (NAPLAN) administered annually in Australia to students in 
Grades 3, 5, 7 and 9 was available from 2013 for Grade 7 and 2012 for Grade 8. 


Analyses 


QS and control group students initial CASS averaged response time and accuracy 
scores, PATM scaled scores at Tl and T2 and NAPLAN numeracy logit scores were 
entered into a Statistical Package for the Social Sciences (SPSS) computer program. 
Speed scores were also entered for the final multiplication lesson for the QS students. 
The statistical analyses were conducted with nine QS students (program attrition of 
one boy in Grade 8) and nine control students (incomplete data for a Grade 7 boy). 


RESULTS 


The median accuracy score measured by the CASS for both groups was 88% prior to 
the intervention and 100% for QS students at the completion of the multiplication 
lessons. Initial CASS speed scores for QS students, shown in Figure 1, ranged from 
1.77 - 4.80 seconds, with a mean of 2.78 secs and SD of 0.96. Control group scores 
ranged from 1.50 - 4.79 seconds with a mean of 2.64 (SD = 0.93). Analysis of Variance 
(ANOVA) revealed no statistically significant differences between the mean speed of 
the two groups before the QS began [F (1,16) <1]. However, there was a statistically 
significant relationship between QS students’ mean speed score prior to and at the end 
(Mean = 1.21 secs, SD 0.34) of the multiplication lessons [F (1, 8) = 39.28, p<0.001]. 


Response time (Seconds) to multiplication 
items 


Time 1 Time 2 
Phase 


Figure 1: Speed scores for QS students prior to and after the multiplication intervention 


Although students were administered different pre and post tests their scaled scores 
can be validly compared as PATM tests are scaled on a single interval scale of 
mathematics achievement through the RASCH measurement model (ACER, 2010). 
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The effect of prior knowledge on PATM scores and QS intervention was tested by 
using students’ NAPLAN scores as a covariate. While NAPLAN scores predicted 
students’ PATM score at T1, it did not serve as a covariate for the treatment effect. 


The difference between the mean PATM scaled scores presented in Figure 2 for QS 
students at Tl of 53.2 (SD = 2.9) and 49.9 (SD = 2.6) for the controls was statistically 
significant (p< 0.005). The difference between the mean PATM scaled scores at T2 of 
55.58 (SD = 3.3) for QS and 51.0 (SD = 4.0) for controls was also statistically 
significant (p< 0.001). Covariance analysis which controls for the baseline score at T1 
showed the group effect remained significant [F (1,15) = 6.5 p< 0.022], with the 
achievement of the QS students increasing more over time compared with the control 
group. Repeated measures interaction approached significance [F(1,15)=6.5, p<0.06]. 


Group 


—- QuickSmart 
—— Control 


Mean PATM Scaled Score 


Figure 2: PATM scaled scores for QS and control students at Tl and T2 
DISCUSSION 


Developing automaticity in cognitive processing 1s a major goal in mathematics for 
students in the early elementary grades. Failure to acquire basic mathematics skills by 
the middle school grades has significant consequences for students who have to 
employ effortful and costly mental strategies to solve tasks that essentially require low 
level knowledge (Hattie & Yates, 2014). Lack of automaticity was evident for both 
groups of students in the initial CASS scores where their mean response latencies were 
reasonably accurate but slow. There was also considerable variability in the response 
speeds of both QS and control group students, shown in Figure 1 for the QS group. The 
statistically significant decrease in the mean response latency for the QS group by the 
end of the multiplication lessons is an important finding as it extends previous studies 
which have reported students to be quicker in fact retrieval and smarter at strategy use 
by the conclusion of the QS intervention (Bellert, 2009; Pegg & Graham, 2009). 


In relation to the second aim of the study, students’ performance on PATM at T1 was 
predicted by their numeracy achievement as measured by the NAPLAN. Both the 
PATM and NAPLAN numeracy test were administered under timed conditions so it is 


PME 2014 5 - 397 


Articles published in the Proceedings are copyrighted by the authors. 


Yates, Lockwood 


likely that lack of fluency would have influenced the performance of both groups 
(Arroyo, Woolf, Royer, Tai & English, 2010). Previous studies have found speed of 
retrieval of mathematics facts to be a significant predictor of middle school students’ 
test performance (Royer, Tronsky, Chan, Jackson, & Marchant, 1999). 


The statistically significantly difference in PATM scores between QS participants and 
control group at T1 is difficult to explain. However, the significant increase in the gap 
between the achievement of the QS and control groups at T2 is a notable finding which 
can be attributed to the QS intervention., There was no statistically significant 
difference in mean CASS response latencies between the two groups prior to the 
intervention, the effect of baseline performance on PATM was controlled for in the 
covariance analysis, QS and control students were paired within their respective Grade 
7 or 8 classrooms and received the same number of mathematics lessons each week, at 
the same time, from the same teachers, with the same textbooks and over the same time 
frame. Further, although the key focus of the additional lessons received by the QS 
group was to improve their understanding and speedy recall of basic multiplication 
facts, through the rehearsal of more sophisticated and efficient strategies which foster 
automatic recall (Bellert, 2009), the increase in their PATM scores 1s also evidence of 
generalisation to learning in other domains of mathematics. This finding extends 
evidence from a previous study in which middle school students had significantly 
higher PATM raw scores after completing the QS program in the four basic processes 
(effect size = 0.65) (Bellert, 2009). The current finding also raises the interesting 
research question of whether it is just as efficacious to administer the multiplication 
section of the program only rather than in its entirety. 


With respect to Aim 1 from the school perspective, the implementation of the 
intervention program in the middle school grades has considerable response costs 
associated with the purchase of the QS program, annual licensing fee, teacher aide and 
supervising teacher training and allocation of teaching time and space within which to 
operate the program. Further, QS participants have to be withdrawn from three lessons 
every week which affects their participation in their other subject areas. QS lessons 
were timetabled to occur on a Monday, Wednesday and Friday to provide the 
opportunity for spaced rather than massed practice (Carpenter, 2014), but it is 
interesting to note that over Terms 3 and 4 (of 10 and 9 weeks duration respectively) 
students completed only a mean of 33 QS lessons over 16.5 hours. While some of the 
discrepancy can be explained by the time taken with the initial CASS testing with both 
groups of students and student absence from school, it was noted on several occasions 
that opportunities for QS lessons to occur were affected by school sanctioned activities 
such as assemblies, excursions, sports days and other events. 


These costs of implementing an intervention in the middle school grades have to be 
considered against the long term effects of not providing any intervention. There is 
considerable research evidence that students’ ability to retrieve basic number facts will 
not improve across the elementary school years without intervention (Gersten, Jordan 
& Flojo, 2005). Further, speed of retrieval is a significant predictor of students’ 
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achievement on mathematics tests throughout their secondary schooling and beyond 
(Royer et al., 1999). While the effects of the numeracy intervention on student work 
samples, self efficacy, anxiety and learned helplessness in mathematics (Yates, 2009b) 
will be considered at the completion of the trial of the QS program in 2014, the results 
thus far nevertheless indicate quite strongly that significant positive changes are 
evident in response latencies, accuracy and achievement in mathematics when students 
are provided with the knowledge of and opportunities to practice more efficient and 
effective basic skills and strategies in a supportive small group environment with 
motivating feedback. 


Acknowledgements 
Assistance from Mrs Pamela Holmes and John Santini is gratefully acknowledged. 
References 


Arroyo, I., Royer, J. M., & Woolf, B. P. (2011). Using an intelligent tutor and math fluency 
training to improve math performance. /nternational Journal of Artificial Intelligence in 
Education, 21, 135-152. 


Arroyo, I., Woolf, B. P., Royer, J. M., Tai, M., & English, S. (2010). Improving math 


learning through intelligent tutoring and basic skills training.. In V. Aleven, J. Kay, & J. 
Mostow (Eds.), JTS 2010, Part 1, LCNS 6094. Berlin, Heidelberg: Springer-Verlag. 


Australian Council for Educational Research. (2010). The Progressive Achievement Tests in 
Mathematics Plus. Melbourne: ACER. 


Bellert, A. (2009). Narrowing the gap: A report on the QuickSmart mathematics intervention. 
Australian Journal of Learning Difficulties, 14(2), 171-183. 


Carpenter, S. K. (2014). Spacing and interleaving of study and practice. In V. A. Benassi, C. 
E. Overson, & C. M. Hakala (Eds.), Applying science of learning in education: Infusing 
psychological science into the curriculum (pp. 131-141). Washington, DC: Division 2, 
American Psychological Association. 


Carr, M., Taasoobshirazi, G., Stroud, R., & Royer, J. M. (2011). Combined fluency and 
cognitive strategies instruction improves mathematics achievement in early elementary 
school. Contemporary Educational Psychology, 36, 323-333. 


Gersten, R., Jordan, N. C., & Flojo, J. R. (2005). Early identification and interventions for 
students with mathematical difficulties. Journal of Learning Disabilities, 38(4), 293-304. 


Graham, L., Bellert, A., Thomas, J.. & Pegg, J. (2007). QuickSmart: A basic skills 
intervention for middle school students with learning difficulties. Journal of Learning 
Disabilities, 40, 410-419. 


Graham, L., & Pegg, J. (2010). Hard data to support the effectiveness of QuickSmart 
numeracy. Learning Difficulties Australia Bulletin, 42(1), 11-13. 


Hattie, J., & Yates, G. C. R. (2014). Visible learning and the science of how we learn. 
Abington, Oxon: Routledge. 


Mayer, R. E. (2006). The role of knowledge in the development of mathematical reasoning. 
In R. L. Sternberg & R. F. Subotnik (Eds.), Optimising student success in school with the 


PME 2014 5 - 399 


Articles published in the Proceedings are copyrighted by the authors. 


Yates, Lockwood 


other three Rs: Reasoning, resilience and responsibility (pp. 59-74). Greenwich, 
Connecticut: Information Age Publishing. 


National Mathematics Advisory Panel. (2008). Foundations for success. The final report of 
the National Mathematics Advisory Panel. Washington, DC: U.S. Department of 
Education. 


Pegg. J. E., & Graham, L. (2007). Addressing the needs of low achieving mathematics 
students: Helping students to trust their heads. Keynote. In K. Milton, H. Reeves, & T. 
Spencer (Eds.), Proc. 21° Conference of the Australian Association of Mathematics 
Teachers (pp. 33-45). Adelaide: The Australian Association of Mathematics Teachers, Inc. 


Pegg. J. E., & Graham, L. (2009). QuickSmart intervention research program data 
2001-2008 full report. Armidale, NSW: University of New England. 


Pegg. J. E., Graham, L., & Bellert, A. (2005). The effects of improved automaticity of basic 
number skills on persistently low-achieving pupils. In H. L. Chick & J. L. Vincent (Eds.), 
Proc. 29" Conf. of the Int. Group for the Psychology of Mathematics Education (Vol. 4, 
pp. 4-49). Melbourne, Australia: PME. 


Royer, J. M. (1996). Cognitive assessment of reading competencies: Diagnosis and 
prescription. In G. D. Pyne (Ed.), Handbook of academic learning (pp. 199-234). San 
Diego, CA: Academic Press. 


Royer, J. M., Tronsky, L. N., Chan, Y., Jackson, S. J., & Marchant, H. G. (1999). Math fact 
retrieval as the cognitive mechanism underlying gender differences in math test 
performance. Contemporary Educational Psychology, 31, 181-266. 

Thomson, S., De Bortoli, L., & Buckley, S. (2013). PISA in brief: highlights from the full 
Australian report: PISA 2012: How Australia measures up. Melbourne: ACER. 


Thomson, S., Hillman, K., & De Bortoli, L. (2013). A teacher’s guide to PISA mathematical 
literacy. Melbourne: ACER. 


Thomson, S., Hillman, K., & Wernert, N. (2012). Monitoring Australian Year 8 student 
achievement internationally: TIMSS 2011. Melbourne: ACER. 


Tronsky, L. T., & Royer, J. M. (2002). Relationships among basic computational 
automaticity, working memory, and complex problem solving: What we know and what 
we need to know. In J. M. Royer (Ed.), Mathematical cognition (pp. 117-146). Greenwich, 
Connecticut: Information Age Publishing. 


Vaughn, S. R., Bos, C. S., & Schumm, J. S. (2000). Teaching exceptional, diverse, and 
at-risk students in the general education classroom. Boston, MA: Allyn & Bacon. 


Westwood, P. (2004). Learning and learning difficulties: A handbook for teachers. 
Melbourne: ACER. 


Yates, S. M. (2009a). The “back to basics” dilemma for middle school teachers. In R. Hunter, 
B. Bicknell, & T. Burgess (Eds.), Crossing divides: Proceedings of the 32" annual 
conference of the Mathematics Education Research Group of Australasia (Vol. 2, pp. 
619-625). Palmerston North, NZ: MERGA. 


Yates, S. M. (2009b). Teacher identification of student learned helplessness in mathematics. 
Mathematics Education Research Journal, 21(3), 86-106. 


5 - 400 PME 2014 


Articles published in the Proceedings are copyrighted by the authors. 


